Slurp is an advanced passive NNTP client for UNIX. It will connect to a remote NNTP server and retrieve articles in a specified set of Usenet newsgroups that have arrived after a particular time (typically the last time it was invoked) for processing by your local news system.
Use this to specify an alternate extension to slurp.<hostname>. Normally Slurp will use the hostname, but this can cause problems on file systems with short filenames.
There are two configuration files used by slurp.
Entries in slurp.sys take the form
This format should be familar to people who have used the C News sys file. Entries for a particular host can be continued on more than one line by using a '\' at the end of the line. e.g.
Slurp is even more picky about the presence of whitespace than C News. It can only appear in comments. Comments begin with a '#' and continue to the end of the line.
Using distributions is not recommended - they're only really included for completeness. Under many NNTP implementations they will result in a large extra load on the server and increase the total amount of time for your connection.
There are 4 possible flags: i, l, r and w, which have the same meaning as the command line options. If present, username and password will be sent to the server using the NNTP simple authorisation protocol when the connection is first made. If present, output sets the directory in which news batches are to be written or, if it begins with a '|', the name of a program to which articles will be piped.
The file slurp.<hostname> contains the time when Slurp last connected to the NNTP server at hostname. If a sublist has been specified with the -s option then this will be appended with a period to the name. Slurp can then use this time to pick up all the articles that have arrived at the server since the last session. It may be followed on subsequent lines by a list of Message-IDs of articles that are to be retrieved from the server in the next session.
Each time Slurp is run and slurp.<hostname> updated, the current slurp.<hostname> will be backed up in the file slurp.<hostname>.o.
When run, Slurp will first retrieve the appropriate newsgroup list, distribution list and start time for the specified server, either from the configuration files or overriding those settings with the command line options.
If the -m option is not set, then the current time will be obtained to use as the start time for the next session. If the -l option is specified, this will be taken from the local machine, otherwise it will be retrieved from the remote server through a call to the tcp time service there. If -i is not specified, then the Message-IDs of any articles which were not retrieved in the last session will be loaded from slurp.<hostname>.
Slurp will now connect to the NNTP server at the remote host. If a username and password for use with the NNTP simple authorisation protocol have been supplied then they will be sent to the server. If the -r option is specified, then a 'MODE READER' command will be sent, to ensure at INN sites that Slurp is talking to nnrpd.
If the -n option has not been specified, a NEWNEWS request will now be issued, asking for all the articles that have arrived in the specified list of newsgroups since the specified time. The server will respond with a list of Message-IDs. If the -h option has been specified or a Message-ID is not already present in the local history file, then it will be stored in memory. If the list of newsgroups is too large to fit on one line (NNTP has a maximum line length of 512 characters) then a series of NEWNEWS requests will be carried out, adding further Message-IDs to the memory list if they are not already present. If there are too many Message-IDs to fit in memory then the overflow will be later written to slurp.<hostname>.
Once the NEWNEWS has been completed, and if the -c option has not been specified, Slurp moves into an article retrieval stage. It will go through the list of Message-IDs in memory and request them in turn from the server, adding each article to the batch of articles being either stored in the incoming news directory or piped to another program, usually rnews. When a batch is found to be larger than the maximum size, it will be submitted to the news system.
Once all the articles have been retrieved, the final batch of articles will be submitted. If the -m option has not been set, then the previously obtained time to use for the next NEWNEWS will be written to slurp.<hostname>. If the -w option has not been set then any uncollected Message-IDs are also written to this file for possible later retrieval.
Statistics on the connection will be logged to syslog (or stderr if syslog is not available). The new article count is the total number of articles that have been submitted to the new system. The duplicate count is how many Message-IDs were found to already exist on the local system. If two NEWNEWS requests are necessary and a message ID was returned by both requests, then it will be included twice in the duplicate count. The missing count is those articles which were in the server's history file but didn't exist as actual article files, usually because they have been cancelled. If configured, the speed of transfer of the article retrieval stage will also be logged.
Slurp returns a series of return codes which may be useful to controlling programs:-